Skip to content

feat(xtest): Lets otdf-sdk-mgr manage platform too#451

Open
dmihalcik-virtru wants to merge 6 commits into
mainfrom
DSPX-3302-02-platform-installer
Open

feat(xtest): Lets otdf-sdk-mgr manage platform too#451
dmihalcik-virtru wants to merge 6 commits into
mainfrom
DSPX-3302-02-platform-installer

Conversation

@dmihalcik-virtru
Copy link
Copy Markdown
Member

@dmihalcik-virtru dmihalcik-virtru commented May 15, 2026

Promotes the OpenTDF platform service to a first-class managed package in otdf-sdk-mgr, mirroring the existing Go/Java/JS SDK CLI flow, and adds a one-shot install scenario entry point.

Summary

Second PR in the five-part stack.

  • platform_installer.py: resolves v0.9.0 to the service/v0.9.0 tag in the opentdf/platform monorepo, creates a git worktree, and runs go build -o xtest/platform/dist/<version>/service ./service.
  • install_helper_scripts(main): mirrors platform/scripts/ into xtest/platform/scripts/. Helper scripts are shared across instances and refreshed on demand.
  • New CLI commands:
    • otdf-sdk-mgr install release platform:<version> (alongside go:, js:, java:)
    • otdf-sdk-mgr install lts platform / install tip platform
    • otdf-sdk-mgr install scripts
    • otdf-sdk-mgr install scenario <path> — installs platform pin + per-KAS pins + encrypt/decrypt SDK union from a scenarios.yaml or instance.yaml, writes <path>.installed.json
  • Container-image platform pins are rejected with a clear v1-not-supported message (no public images today).

Stack

  1. (merged into base) Shared schema — chore(xtest): Shared Scenario/Instance Pydantic schema in otdf-sdk-mgr #450
  2. This PR — Platform installer + install scenario
  3. otdf-local multi-instance refactor + new CLI subcommands
  4. xtest/conftest.py integration (--scenario, --instance)
  5. Claude plugin

Test plan

  • cd otdf-sdk-mgr && uv run pytest tests/ → 57 passing (existing 51 + new 6 from PR 1)
  • uv run otdf-sdk-mgr install --help shows the new scenario and scripts commands

Jira: https://virtru.atlassian.net/browse/DSPX-3302

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added scenario installation command (writes consumable manifests; handles partial failures)
    • Added scripts command to refresh shared platform helper scripts
    • lts/tip/release now support installing the platform
  • Documentation

    • Multiple AGENTS.md updates clarifying repository layout, install workflow, and platform dependency notes
    • CLI/help text updated with platform examples
  • Tests

    • Added end-to-end scenario tests and platform-ref resolution tests

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 15, 2026

📝 Walkthrough

Walkthrough

Adds a platform installer module, exposes a public YAML loader, updates install CLI commands to handle platform targets (and a new scripts subcommand), introduces a scenario CLI command that installs platform/KAS/SDKs from YAML and writes manifests, and includes unit and integration tests plus docs updates.

Changes

Platform Installation and Scenario Orchestration

Layer / File(s) Summary
Schema YAML loading refactoring
otdf-sdk-mgr/src/otdf_sdk_mgr/schema.py
load_yaml_mapping is exposed as a public YAML loader and load_scenario/load_instance plus the schema CLI now use it.
Platform installer module
otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py, otdf-sdk-mgr/tests/test_platform_installer.py
New module to locate the platform workspace, manage bare repo and detached worktrees, resolve refs, build the Go service binary into dist/<version>/, install helper scripts from a branch, and list available versions. Unit tests cover ref normalization.
CLI install command updates
otdf-sdk-mgr/src/otdf_sdk_mgr/cli_install.py
lts, tip, and release now accept platform targets and invoke the platform installer; scripts subcommand refreshes helper scripts; internal helpers register the scenario subcommand and map platform errors to typer.Exit.
Scenario CLI command and integration tests
otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py, otdf-sdk-mgr/tests/test_cli_scenario.py
New install_scenario_cmd parses scenario/instance YAML, installs pinned platform/KAS artifacts and encrypt/decrypt SDKs, optionally refreshes helper scripts, and emits a {stem}.installed.json manifest; tests verify success and partial-manifest failure modes.

Developer Workflow & Repo Docs

Layer / File(s) Summary
Repository AGENTS and local dependency notes
AGENTS.md, otdf-sdk-mgr/AGENTS.md, otdf-local/AGENTS.md, xtest/AGENTS.md, otdf-sdk-mgr/CLAUDE.md
Update repository layout and closing guidance in top-level AGENTS.md; add agent guide for otdf-sdk-mgr; note otdf-local dependency on otdf-sdk-mgr installs; adjust xtest link to otdf-sdk-mgr/AGENTS.md; add short CLAUDE.md content.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • opentdf/tests#450: Introduces shared Pydantic models and helpers (Scenario, Instance, scenario_to_pytest_sdks) used by the new cli_scenario command.

Suggested reviewers

  • pflynn-virtru
  • sujankota
  • sievdokymov-virtru

Poem

🐰 A rabbit hops over branches of code,
Building worktrees where platform binaries grow.
From YAML to manifests, installs take flight,
Scripts refreshed, tests green in the night.
Hooray — lint and run, and everything's right!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 63.16% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(xtest): Lets otdf-sdk-mgr manage platform too' directly aligns with the main objective of the changeset: promoting the OpenTDF platform service to a first-class managed package in otdf-sdk-mgr.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch DSPX-3302-02-platform-installer

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for installing the OpenTDF platform service and scenario-driven installations. Key changes include the addition of a platform_installer module that manages source builds via git worktrees, a new cli_scenario module for manifest-based installs, and updates to existing CLI commands to handle the "platform" target. Review feedback highlights a bug in updating git worktrees from bare repositories, identifies code duplication in the CLI logic, suggests optimizing YAML parsing to avoid redundant reads, and recommends allowing real-time output for long-running build processes to improve user experience.

Comment thread otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py Outdated
Comment on lines +52 to +65
requested = sdks or ALL_SDKS
sdk_targets = [s for s in requested if s != "platform"]
if "platform" in requested:
version = LTS_VERSIONS.get("platform")
if version is None:
typer.echo("Warning: no LTS version defined for platform; skipping", err=True)
else:
try:
install_platform_release(version)
except PlatformInstallError as e:
typer.echo(f"Error: {e}", err=True)
raise typer.Exit(1)
if sdk_targets:
cmd_lts(sdk_targets)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic for handling the platform target is duplicated here and in the tip command (lines 82-91). This pattern makes the CLI code harder to maintain. Consider refactoring this into a shared helper function or extending the cmd_lts/cmd_tip functions to handle the platform service internally, similar to how other SDKs are handled.

Comment thread otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py Outdated
Comment thread otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py Outdated
@github-actions
Copy link
Copy Markdown

@dmihalcik-virtru dmihalcik-virtru force-pushed the DSPX-3302-02-platform-installer branch from c6a7895 to ebc0c15 Compare May 15, 2026 16:35
@dmihalcik-virtru dmihalcik-virtru force-pushed the DSPX-3302-02-platform-installer branch from ebc0c15 to 14e5c1e Compare May 15, 2026 16:57
dmihalcik-virtru added a commit that referenced this pull request May 21, 2026
#450)

## Summary

First PR in a five-part stack that introduces a multi-instance test
harness and a Claude plugin for OpenTDF bug reproduction. This PR adds
*only* the shared Pydantic schema in `otdf-sdk-mgr` — no consumers yet.

- Adds `otdf_sdk_mgr.schema` with v2 models: `Scenario`, `Instance`,
`PlatformPin`, `KasPin`, `SdkPin`, `ScenarioSdks`, `Suite`, etc.
- `ScenarioSdks.encrypt` / `.decrypt` mirror xtest's existing
`--sdks-encrypt` / `--sdks-decrypt` convention so a→b-only scenarios are
first-class.
- `python -m otdf_sdk_mgr.schema validate <path>` validates either a
Scenario or an Instance file based on its `kind:`.
- Adds `pydantic` + `ruamel.yaml` to `otdf-sdk-mgr/pyproject.toml`.
- 6 unit tests covering round-trips, pin invariants, and unknown-field
rejection.

## Stack

1. [**This PR**](#450) — Shared
schema
2. [Platform installer + `install
scenario`](#451) in `otdf-sdk-mgr`
(builds on this)
3. `otdf-local` [multi-instance
refactor](#452) + new CLI
subcommands
4. `xtest/conftest.py`
[integration](#453) (`--scenario`,
`--instance`)
5. [Claude plugin](#454)
(`.claude/skills/`, settings, plugin manifest)
6. #455

## Test plan

- [x] `cd otdf-sdk-mgr && uv run pytest tests/test_schema.py` — all 6
pass
- [x] `uv run python -m otdf_sdk_mgr.schema validate <path>` accepts a
valid scenarios.yaml and rejects unknown fields

Jira: https://virtru.atlassian.net/browse/DSPX-3302

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added schema validation for OpenTDF Scenario and Instance YAML
configurations with a new CLI command.
* Introduced strict validation with cross-field constraints for SDK and
platform configurations.

* **Documentation**
  * Updated supported container formats from `nano` to `ztdf-ecwrap`.

* **Dependencies**
* Updated core package dependencies to support enhanced validation
capabilities.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/opentdf/tests/pull/450?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Base automatically changed from DSPX-3302-01-shared-schema to main May 21, 2026 15:21
@dmihalcik-virtru dmihalcik-virtru force-pushed the DSPX-3302-02-platform-installer branch from 14e5c1e to 9993b12 Compare May 21, 2026 15:38
@dmihalcik-virtru dmihalcik-virtru changed the title [DSPX-3302] (2/5) Manage platform service + install scenario in otdf-sdk-mgr feat(otdf-sdk-mgr): Lets otdf-sdk-mgr manage platform too May 21, 2026
@github-actions
Copy link
Copy Markdown

@github-actions
Copy link
Copy Markdown

X-Test Failure Report

@dmihalcik-virtru dmihalcik-virtru changed the title feat(otdf-sdk-mgr): Lets otdf-sdk-mgr manage platform too feat(xtest): Lets otdf-sdk-mgr manage platform too May 21, 2026
@github-actions
Copy link
Copy Markdown

@dmihalcik-virtru dmihalcik-virtru marked this pull request as ready for review May 21, 2026 16:24
@dmihalcik-virtru dmihalcik-virtru requested review from a team as code owners May 21, 2026 16:24
@github-actions
Copy link
Copy Markdown

X-Test Failure Report

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
otdf-sdk-mgr/tests/test_cli_scenario.py (1)

107-126: ⚡ Quick win

Add a validation-error CLI test case.

Please add one test for schema-invalid (but syntactically valid) YAML to assert install_scenario_cmd exits cleanly with typer.Exit(1) and no traceback.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@otdf-sdk-mgr/tests/test_cli_scenario.py` around lines 107 - 126, Add a new
test in tests/test_cli_scenario.py that submits a syntactically valid but
schema-invalid YAML to install_scenario_cmd and asserts it exits cleanly with
typer.Exit(status=1) and no traceback; specifically, create a tmp_path scenario
file containing SCENARIO_YAML with a deliberate schema violation, call
install_scenario_cmd(scenario_path, ...) in a pytest.raises(typer.Exit) context
and assert the raised exception has exit_code == 1 and that no exception other
than typer.Exit is propagated (mirroring the pattern used in
test_install_scenario_writes_partial_manifest_on_failure), referencing
install_scenario_cmd and the tmp_path-based scenario file to locate where to add
the test.
otdf-sdk-mgr/tests/test_platform_installer.py (1)

8-23: ⚡ Quick win

Add a regression case for container-image-style refs.

Please add a parametrized case asserting container-image inputs are rejected (clear v1-not-supported error), so this behavior stays locked.

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py`:
- Around line 57-72: The code only catches YAMLError from load_yaml_mapping but
lets other parse/validation errors escape; update the block around
load_yaml_mapping, the raw kind extraction, and the calls to
Scenario.model_validate and Instance.model_validate to also handle non-YAML
failures by validating that raw is a mapping (isinstance(raw, dict)) and
wrapping model_validate calls in a try/except that catches validation/parse/type
errors (e.g., ValueError/TypeError and the validation error your pydantic layer
raises) and in the except branch call typer.echo with a clear message including
the exception and then raise typer.Exit(1); ensure instance and scenario
variables are only used after successful validation.

---

Nitpick comments:
In `@otdf-sdk-mgr/tests/test_cli_scenario.py`:
- Around line 107-126: Add a new test in tests/test_cli_scenario.py that submits
a syntactically valid but schema-invalid YAML to install_scenario_cmd and
asserts it exits cleanly with typer.Exit(status=1) and no traceback;
specifically, create a tmp_path scenario file containing SCENARIO_YAML with a
deliberate schema violation, call install_scenario_cmd(scenario_path, ...) in a
pytest.raises(typer.Exit) context and assert the raised exception has exit_code
== 1 and that no exception other than typer.Exit is propagated (mirroring the
pattern used in test_install_scenario_writes_partial_manifest_on_failure),
referencing install_scenario_cmd and the tmp_path-based scenario file to locate
where to add the test.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7eb48abc-654d-4040-9cfe-925dec493035

📥 Commits

Reviewing files that changed from the base of the PR and between cd004bc and ec1f655.

📒 Files selected for processing (7)
  • AGENTS.md
  • otdf-sdk-mgr/src/otdf_sdk_mgr/cli_install.py
  • otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py
  • otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py
  • otdf-sdk-mgr/src/otdf_sdk_mgr/schema.py
  • otdf-sdk-mgr/tests/test_cli_scenario.py
  • otdf-sdk-mgr/tests/test_platform_installer.py

Comment on lines +57 to +72
try:
raw = load_yaml_mapping(path)
except YAMLError as e:
typer.echo(f"Error: {path} is not valid YAML: {e}", err=True)
raise typer.Exit(1)

kind = raw.get("kind") if isinstance(raw.get("kind"), str) else None
scenario: Scenario | None = None
if kind == "Scenario":
scenario = Scenario.model_validate(raw)
instance = scenario.instance
elif kind == "Instance":
instance = Instance.model_validate(raw)
else:
typer.echo(f"Error: {path} has unknown kind {kind!r}", err=True)
raise typer.Exit(1)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Handle non-YAML parse/validation failures as CLI exits.

Line 57-72 only handles YAML syntax errors. Read errors, top-level type errors, and schema validation failures currently escape as uncaught exceptions.

Proposed fix
 import typer
+from pydantic import ValidationError
@@
     try:
         raw = load_yaml_mapping(path)
-    except YAMLError as e:
+    except (OSError, YAMLError, ValueError) as e:
         typer.echo(f"Error: {path} is not valid YAML: {e}", err=True)
         raise typer.Exit(1)
@@
-    if kind == "Scenario":
-        scenario = Scenario.model_validate(raw)
-        instance = scenario.instance
-    elif kind == "Instance":
-        instance = Instance.model_validate(raw)
-    else:
-        typer.echo(f"Error: {path} has unknown kind {kind!r}", err=True)
-        raise typer.Exit(1)
+    try:
+        if kind == "Scenario":
+            scenario = Scenario.model_validate(raw)
+            instance = scenario.instance
+        elif kind == "Instance":
+            instance = Instance.model_validate(raw)
+        else:
+            typer.echo(f"Error: {path} has unknown kind {kind!r}", err=True)
+            raise typer.Exit(1)
+    except ValidationError as e:
+        typer.echo(f"Error: invalid manifest {path}: {e}", err=True)
+        raise typer.Exit(1)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
try:
raw = load_yaml_mapping(path)
except YAMLError as e:
typer.echo(f"Error: {path} is not valid YAML: {e}", err=True)
raise typer.Exit(1)
kind = raw.get("kind") if isinstance(raw.get("kind"), str) else None
scenario: Scenario | None = None
if kind == "Scenario":
scenario = Scenario.model_validate(raw)
instance = scenario.instance
elif kind == "Instance":
instance = Instance.model_validate(raw)
else:
typer.echo(f"Error: {path} has unknown kind {kind!r}", err=True)
raise typer.Exit(1)
try:
raw = load_yaml_mapping(path)
except (OSError, YAMLError, ValueError) as e:
typer.echo(f"Error: {path} is not valid YAML: {e}", err=True)
raise typer.Exit(1)
kind = raw.get("kind") if isinstance(raw.get("kind"), str) else None
scenario: Scenario | None = None
try:
if kind == "Scenario":
scenario = Scenario.model_validate(raw)
instance = scenario.instance
elif kind == "Instance":
instance = Instance.model_validate(raw)
else:
typer.echo(f"Error: {path} has unknown kind {kind!r}", err=True)
raise typer.Exit(1)
except ValidationError as e:
typer.echo(f"Error: invalid manifest {path}: {e}", err=True)
raise typer.Exit(1)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py` around lines 57 - 72, The code
only catches YAMLError from load_yaml_mapping but lets other parse/validation
errors escape; update the block around load_yaml_mapping, the raw kind
extraction, and the calls to Scenario.model_validate and Instance.model_validate
to also handle non-YAML failures by validating that raw is a mapping
(isinstance(raw, dict)) and wrapping model_validate calls in a try/except that
catches validation/parse/type errors (e.g., ValueError/TypeError and the
validation error your pydantic layer raises) and in the except branch call
typer.echo with a clear message including the exception and then raise
typer.Exit(1); ensure instance and scenario variables are only used after
successful validation.

Comment on lines +91 to +104
def _resolve_platform_ref(version_or_ref: str) -> str:
"""Turn a user-supplied version into the actual git ref to checkout.

`v0.9.0` → `service/v0.9.0` (matches SDK_TAG_INFIXES["platform"]).
A ref that already contains `/`, a hex SHA, or `main` is returned as-is.
"""
infix = SDK_TAG_INFIXES.get("platform", "service")
if "/" in version_or_ref or version_or_ref in ("main", "HEAD"):
return version_or_ref
if len(version_or_ref) in (40, 64) and all(
c in "0123456789abcdef" for c in version_or_ref.lower()
):
return version_or_ref
return f"{infix}/{normalize_version(version_or_ref)}"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Reject container-image refs early with a clear error.

Line 98 currently passes through any string containing /, so container-image pins fall through to git commands and fail with a generic error instead of the intended v1-not-supported message.

Proposed fix
 def _resolve_platform_ref(version_or_ref: str) -> str:
@@
-    if "/" in version_or_ref or version_or_ref in ("main", "HEAD"):
+    if ("@" in version_or_ref or ":" in version_or_ref) and "/" in version_or_ref:
+        raise PlatformInstallError(
+            "container-image platform pins are not supported in v1; "
+            "use a git ref like v0.9.0 or service/v0.9.0"
+        )
+    if "/" in version_or_ref or version_or_ref in ("main", "HEAD"):
         return version_or_ref

dmihalcik-virtru added a commit that referenced this pull request May 22, 2026
## Summary

Improves the agent-facing documentation across the repository so future
Claude Code sessions land on accurate context faster. All changes are
docs-only — no code touched.

- **Root `AGENTS.md`**: new Repository Layout table at the top; new
"Before Committing Python Changes" section requiring `uv run ruff
check`, `uv run ruff format`, and `uv run pyright`, with the rationale
that `uvx` strips the project venv and breaks pyright import resolution;
trimmed the duplicated "Summary → Preferred Workflow" block.
- **`otdf-local/AGENTS.md`**: flagged the manual-keys YAML block as an
emergency fallback that may drift from the current platform schema.
- **`xtest/AGENTS.md`** (new): test-suite layout, custom pytest options
reference, audit-log fixture quick reference (`--no-audit-logs` vs
`DISABLE_AUDIT_ASSERTIONS`), and test-authoring guidance.
`xtest/CLAUDE.md` symlinks to it, matching the root convention.

Scoped intentionally to content that's accurate on `main` today. A
follow-up PR will add `otdf-sdk-mgr/AGENTS.md` (covering the platform
installer and scenario workflows) once #451 merges.

## Test plan
- [x] No source files modified — verified with `git diff --stat`
- [x] `grep` for forward references to feature-branch-only code
(`platform_installer`, `cli_scenario`, `install platform`, `install
scenario`) returns no matches in these files
- [x] All command snippets are commands that work against `main` today

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated onboarding guidance for the test framework, including
repository layout and required pre-commit linting and type-checking
procedures
* Added comprehensive documentation for the cross-client integration
test suite with pytest modules, fixtures, and custom options
* Enhanced Golden Key Auto-Configuration documentation with emergency
fallback configuration details

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/opentdf/tests/pull/459?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
dmihalcik-virtru and others added 5 commits May 21, 2026 21:45
`install scenario` could not run as written: it iterated `ScenarioSdks.union()`
as a dict (it returns a list) and passed a `source=` kwarg `install_release`
does not accept. The emitted `installed.json` shape also did not match what
`scenario_to_pytest_sdks` reads (per-role lists, not sdk-name-keyed dict),
so even the platform-only path produced a manifest no downstream tool
could consume.

Source fixes:
- cli_scenario.py: iterate `union()` as the list it is, cache installs by
  (sdk, version, source), emit role-keyed lists matching the reader's
  expected shape; on failure write a partial manifest with `status=partial`
  so half-installed dist trees are diagnosable. Catch YAMLError in
  `_peek_kind` to surface a clean typer error.
- platform_installer.py: `_git_rev_parse` raises on failure instead of
  silently writing an empty `sha=` into `.version`. Missing `scripts/`
  raises instead of warning-and-continuing. SHA passthrough heuristic
  tightened from `>=7` chars to exactly 40 (SHA-1) or 64 (SHA-256), so
  ambiguous short tags like `abc1234` no longer skip the `service/`
  prefix. Dropped a docstring fragment pointing to a planning doc that
  won't exist post-merge.
- cli_install.py: dropped a docstring whose "deferred import" claim was
  false (the registration runs at module import). `lts platform` with no
  pinned version now exits 1 instead of warning-and-exit-0.

Tests:
- test_platform_installer.py: parametrized cases for `_resolve_platform_ref`
  covering version normalization, branch passthrough, the tightened hex
  heuristic, and SHA-1/SHA-256 passthrough.
- test_cli_scenario.py: end-to-end smoke that mocks the installers and
  asserts the produced manifest is round-trip consumable by
  `scenario_to_pytest_sdks`. This is the gating test that would have
  caught the original bug.

79 passing (was 67).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- platform_installer: fix worktree update from bare repo (no `origin`
  remote exists), use `git reset --hard <branch>` instead of `git pull`
- platform_installer: stop swallowing subprocess output so long-running
  `go build`/`git clone` progress is visible to the user
- cli_install: extract `_install_platform_or_exit` to dedupe platform
  handling across `lts`, `tip`, and `release`
- cli_scenario: parse manifest YAML once and dispatch by `kind`, instead
  of peeking + re-parsing in each loader

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…typing

- AGENTS.md: add "Before Committing Python Changes" section requiring
  `uv run ruff check`, `uv run ruff format`, and `uv run pyright` on any
  touched Python package before commit. Explicitly call out that `uvx`
  must NOT be used for pyright (isolated env can't see project deps, so
  every project import becomes a spurious "could not be resolved" error).
- cli_scenario: split the single `dict[str, object]` install record into
  per-section typed containers (`installed_platform`, `installed_kas`,
  `installed_sdks`) assembled at write time via a `_snapshot()` helper.
  Fixes pre-existing pyright `__setitem__ ... not defined on object`
  errors at the nested writes; on-disk JSON shape is unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Root AGENTS.md: add a Repository Layout table near the top, correct
  the `platform/` description (it's installed by `otdf-sdk-mgr install`,
  not committed source), and trim the duplicated "Summary → Preferred
  Workflow" block that restated the body.
- otdf-local/AGENTS.md: lead with the dependency on `otdf-sdk-mgr`
  (otdf-local launches the binaries the installer produces). Mark the
  manual-keys YAML block as an emergency fallback that may drift.
- otdf-sdk-mgr/AGENTS.md (new): operational guide for the installer —
  subcommand layout, bare-clone-worktree gotchas (no `origin` remote,
  namespaced `service/vX.Y.Z` tags, unbuffered subprocess output),
  pattern for adding a new subcommand.
- xtest/AGENTS.md (new): test-suite layout, custom pytest options,
  audit-log fixture quick reference, authoring guidance.
- otdf-sdk-mgr/CLAUDE.md, xtest/CLAUDE.md: symlinks to AGENTS.md to
  match the repo convention.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@dmihalcik-virtru dmihalcik-virtru force-pushed the DSPX-3302-02-platform-installer branch from ec1f655 to 13b5c96 Compare May 22, 2026 01:46
@sonarqubecloud
Copy link
Copy Markdown

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (2)
otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py (1)

57-72: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Handle non-YAML and schema-validation failures as CLI exits.

Only YAMLError is caught here. Read/type/validation failures can still escape with a traceback instead of a clean CLI error + exit code 1.

Proposed fix
+from pydantic import ValidationError
@@
-    from ruamel.yaml.error import YAMLError
+    from ruamel.yaml.error import YAMLError
@@
     try:
         raw = load_yaml_mapping(path)
-    except YAMLError as e:
+    except (OSError, YAMLError, ValueError) as e:
         typer.echo(f"Error: {path} is not valid YAML: {e}", err=True)
         raise typer.Exit(1)
@@
-    if kind == "Scenario":
-        scenario = Scenario.model_validate(raw)
-        instance = scenario.instance
-    elif kind == "Instance":
-        instance = Instance.model_validate(raw)
-    else:
-        typer.echo(f"Error: {path} has unknown kind {kind!r}", err=True)
-        raise typer.Exit(1)
+    try:
+        if kind == "Scenario":
+            scenario = Scenario.model_validate(raw)
+            instance = scenario.instance
+        elif kind == "Instance":
+            instance = Instance.model_validate(raw)
+        else:
+            typer.echo(f"Error: {path} has unknown kind {kind!r}", err=True)
+            raise typer.Exit(1)
+    except ValidationError as e:
+        typer.echo(f"Error: invalid manifest {path}: {e}", err=True)
+        raise typer.Exit(1)

As per coding guidelines "otdf-sdk-mgr/**/cli_*.py: Wrap library exceptions (InstallError, PlatformInstallError) at the CLI boundary and exit with typer.Exit(1) when adding new subcommands."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py` around lines 57 - 72, Wrap
both the YAML loading and model validation so non-YAML and schema/read/type
failures produce a clean CLI exit: surround the calls to
load_yaml_mapping(path), Scenario.model_validate(raw) and
Instance.model_validate(raw) with a try/except that catches validation/read
errors (e.g., ValidationError, FileNotFoundError and other Exceptions at the CLI
boundary) and on error call typer.echo with a clear message including the
exception and then raise typer.Exit(1); keep using the existing kind dispatch
and ensure the original YAMLError handling remains but add the broader/specific
exception handling around model_validate and loading to avoid tracebacks.
otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py (1)

98-104: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Reject container-image refs before git resolution.

Inputs like ghcr.io/opentdf/platform:vX currently pass through because they contain /, so users get a later git failure instead of the intended v1-not-supported message.

Proposed fix
 def _resolve_platform_ref(version_or_ref: str) -> str:
@@
+    if (":" in version_or_ref or "@" in version_or_ref) and "/" in version_or_ref:
+        raise PlatformInstallError(
+            "container-image platform pins are not supported in v1; "
+            "use a git ref like v0.9.0 or service/v0.9.0"
+        )
     if "/" in version_or_ref or version_or_ref in ("main", "HEAD"):
         return version_or_ref
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py` around lines 98 - 104,
The current check returns version_or_ref whenever it contains "/", which wrongly
treats container image refs like "ghcr.io/org/image:tag" as git refs; update the
condition so it only returns early for git-like paths and not container
images—i.e., replace the simple `if "/" in version_or_ref or version_or_ref in
("main", "HEAD"):` with a check that first rejects container-image patterns
(implement a helper like looks_like_container_image(version_or_ref) that detects
a registry domain in the first segment (contains a dot, e.g. "ghcr.io"), or an
image tag/digest (contains ":" after the last "/" or "@" digest)), and only
return version_or_ref for "/" cases when looks_like_container_image(...) is
false; keep the existing hex-sha length check and the final `return
f"{infix}/{normalize_version(version_or_ref)}"` as-is so container-image refs
fall through to the git resolution path and trigger the correct v1-not-supported
behavior.
🧹 Nitpick comments (1)
otdf-sdk-mgr/tests/test_cli_scenario.py (1)

107-126: ⚡ Quick win

Add a regression test for invalid manifest handling.

Please add a case where schema validation fails (or YAML/top-level mapping is invalid) and assert the command exits via typer.Exit(1) without an unhandled traceback.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@otdf-sdk-mgr/tests/test_cli_scenario.py` around lines 107 - 126, Add a new
regression test in tests/test_cli_scenario.py that supplies an invalid scenario
manifest (e.g., malformed YAML or wrong top-level type) and invokes
install_scenario_cmd, asserting it raises typer.Exit with exit code 1 and does
not propagate a traceback; reuse the test pattern in
test_install_scenario_writes_partial_manifest_on_failure to create a tmp_path
scenario file, patch dependencies used by install_scenario_cmd (like
otdf_sdk_mgr.cli_scenario.install_platform_release, install_helper_scripts,
install_release) as needed so the failure is triggered solely by manifest/schema
validation, and assert the command exits via typer.Exit(1) rather than raising
other exceptions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py`:
- Around line 116-121: The current early return when worktree.exists() reuses
potentially stale files; instead, before returning call _run to refresh the
worktree from the bare repo: perform a git fetch (using --git-dir and
--work-tree with bare and worktree) and then a git reset --hard to the desired
ref (ref) so the checked-out files match the fetched ref; keep using the same
helpers/variables (worktree, bare, ref, _run) and only skip the add call when
the directory already exists after refreshing.

In `@xtest/AGENTS.md`:
- Around line 18-20: Remove the unresolved merge-conflict markers from the
AGENTS.md table: delete the conflict marker line starting with "<<<<<<< HEAD"
and any matching "=======" / ">>>>>>> ..." lines, ensure the two affected rows
(the SDK CLI builds row and the test.env row) are merged into a single
well-formed Markdown table row each, and restore proper table pipe/spacing so
the table renders correctly (look for the lines containing
"`sdk/{go,java,js}/dist/<version>/`" and "`test.env`" to locate the exact rows
to fix).

---

Duplicate comments:
In `@otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py`:
- Around line 57-72: Wrap both the YAML loading and model validation so non-YAML
and schema/read/type failures produce a clean CLI exit: surround the calls to
load_yaml_mapping(path), Scenario.model_validate(raw) and
Instance.model_validate(raw) with a try/except that catches validation/read
errors (e.g., ValidationError, FileNotFoundError and other Exceptions at the CLI
boundary) and on error call typer.echo with a clear message including the
exception and then raise typer.Exit(1); keep using the existing kind dispatch
and ensure the original YAMLError handling remains but add the broader/specific
exception handling around model_validate and loading to avoid tracebacks.

In `@otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py`:
- Around line 98-104: The current check returns version_or_ref whenever it
contains "/", which wrongly treats container image refs like
"ghcr.io/org/image:tag" as git refs; update the condition so it only returns
early for git-like paths and not container images—i.e., replace the simple `if
"/" in version_or_ref or version_or_ref in ("main", "HEAD"):` with a check that
first rejects container-image patterns (implement a helper like
looks_like_container_image(version_or_ref) that detects a registry domain in the
first segment (contains a dot, e.g. "ghcr.io"), or an image tag/digest (contains
":" after the last "/" or "@" digest)), and only return version_or_ref for "/"
cases when looks_like_container_image(...) is false; keep the existing hex-sha
length check and the final `return
f"{infix}/{normalize_version(version_or_ref)}"` as-is so container-image refs
fall through to the git resolution path and trigger the correct v1-not-supported
behavior.

---

Nitpick comments:
In `@otdf-sdk-mgr/tests/test_cli_scenario.py`:
- Around line 107-126: Add a new regression test in tests/test_cli_scenario.py
that supplies an invalid scenario manifest (e.g., malformed YAML or wrong
top-level type) and invokes install_scenario_cmd, asserting it raises typer.Exit
with exit code 1 and does not propagate a traceback; reuse the test pattern in
test_install_scenario_writes_partial_manifest_on_failure to create a tmp_path
scenario file, patch dependencies used by install_scenario_cmd (like
otdf_sdk_mgr.cli_scenario.install_platform_release, install_helper_scripts,
install_release) as needed so the failure is triggered solely by manifest/schema
validation, and assert the command exits via typer.Exit(1) rather than raising
other exceptions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 073e54ad-8ca6-4169-88fd-e4dac6764fde

📥 Commits

Reviewing files that changed from the base of the PR and between ec1f655 and 13b5c96.

📒 Files selected for processing (11)
  • AGENTS.md
  • otdf-local/AGENTS.md
  • otdf-sdk-mgr/AGENTS.md
  • otdf-sdk-mgr/CLAUDE.md
  • otdf-sdk-mgr/src/otdf_sdk_mgr/cli_install.py
  • otdf-sdk-mgr/src/otdf_sdk_mgr/cli_scenario.py
  • otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py
  • otdf-sdk-mgr/src/otdf_sdk_mgr/schema.py
  • otdf-sdk-mgr/tests/test_cli_scenario.py
  • otdf-sdk-mgr/tests/test_platform_installer.py
  • xtest/AGENTS.md
✅ Files skipped from review due to trivial changes (2)
  • otdf-sdk-mgr/CLAUDE.md
  • otdf-sdk-mgr/AGENTS.md

Comment on lines +116 to +121
if worktree.exists():
print(f"Worktree already exists at {worktree}; reusing.")
return worktree
print(f"Adding worktree at {worktree} for ref {ref}...")
_run(["git", f"--git-dir={bare}", "worktree", "add", "--detach", str(worktree), ref])
return worktree
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Refresh existing worktrees before reuse.

Reusing an existing worktree without resetting it can build stale code for moving refs (notably main), even after the bare repo fetch.

Proposed fix
 def _ensure_worktree(ref: str) -> Path:
@@
     worktree = _worktree_path_for(ref)
     if worktree.exists():
-        print(f"Worktree already exists at {worktree}; reusing.")
+        print(f"Worktree already exists at {worktree}; resetting to {ref}.")
+        _run(["git", "-C", str(worktree), "reset", "--hard", ref])
         return worktree
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@otdf-sdk-mgr/src/otdf_sdk_mgr/platform_installer.py` around lines 116 - 121,
The current early return when worktree.exists() reuses potentially stale files;
instead, before returning call _run to refresh the worktree from the bare repo:
perform a git fetch (using --git-dir and --work-tree with bare and worktree) and
then a git reset --hard to the desired ref (ref) so the checked-out files match
the fetched ref; keep using the same helpers/variables (worktree, bare, ref,
_run) and only skip the add call when the directory already exists after
refreshing.

Comment thread xtest/AGENTS.md
Comment on lines +18 to 20
<<<<<<< HEAD
| `sdk/{go,java,js}/dist/<version>/` | SDK CLI builds. Installed by `otdf-sdk-mgr install` (see `../otdf-sdk-mgr/AGENTS.md`). |
| `test.env` | Default endpoint and client-credential env vars. Source with `set -a && source test.env && set +a`. |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Remove unresolved merge-conflict artifacts from the table.

Line 18 includes a conflict marker (<<<<<<< HEAD), which indicates an incomplete merge and breaks Markdown table structure/linting.

Suggested fix
-<<<<<<< HEAD
 | `sdk/{go,java,js}/dist/<version>/` | SDK CLI builds. Installed by `otdf-sdk-mgr install` (see `../otdf-sdk-mgr/AGENTS.md`). |
 | `test.env` | Default endpoint and client-credential env vars. Source with `set -a && source test.env && set +a`. |
As per coding guidelines: "`xtest/**/AGENTS.md`: Review AGENTS.md for coding standards and guidelines".
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
<<<<<<< HEAD
| `sdk/{go,java,js}/dist/<version>/` | SDK CLI builds. Installed by `otdf-sdk-mgr install` (see `../otdf-sdk-mgr/AGENTS.md`). |
| `test.env` | Default endpoint and client-credential env vars. Source with `set -a && source test.env && set +a`. |
| `sdk/{go,java,js}/dist/<version>/` | SDK CLI builds. Installed by `otdf-sdk-mgr install` (see `../otdf-sdk-mgr/AGENTS.md`). |
| `test.env` | Default endpoint and client-credential env vars. Source with `set -a && source test.env && set +a`. |
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 18-18: Table pipe style
Expected: leading_and_trailing; Actual: no_leading_or_trailing; Missing leading pipe

(MD055, table-pipe-style)


[warning] 18-18: Table pipe style
Expected: leading_and_trailing; Actual: no_leading_or_trailing; Missing trailing pipe

(MD055, table-pipe-style)


[warning] 18-18: Table column count
Expected: 2; Actual: 1; Too few cells, row will be missing data

(MD056, table-column-count)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@xtest/AGENTS.md` around lines 18 - 20, Remove the unresolved merge-conflict
markers from the AGENTS.md table: delete the conflict marker line starting with
"<<<<<<< HEAD" and any matching "=======" / ">>>>>>> ..." lines, ensure the two
affected rows (the SDK CLI builds row and the test.env row) are merged into a
single well-formed Markdown table row each, and restore proper table
pipe/spacing so the table renders correctly (look for the lines containing
"`sdk/{go,java,js}/dist/<version>/`" and "`test.env`" to locate the exact rows
to fix).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant